在本文中,我们描述了我们参与Case-2022的子任务1,即与休闲新闻语料库的事件因果关系识别。我们通过在少数带注释的示例(即几次配置)上利用一组简单但互补的技术来解决因果关系识别(CRI)任务。我们遵循一种基于迅速的预测方法,用于微调LMS,其中CRI任务被视为掩盖语言建模问题(MLM)。这种方法允许LMS在MLM问题上进行本地预先训练,可以直接生成对CRI特异性提示的文本响应。我们将此方法的性能与在整个数据集中训练的集合技术进行比较。我们表现​​最佳的提交仅接受了每班256个实例,整个数据集的一小部分培训,但能够获得第二好的精度(0.82),第三好的精度(0.82)和F1得分。 (0.85)非常接近获胜者团队(0.86)的报道。
translated by 谷歌翻译
In this paper, we perform an exhaustive evaluation of different representations to address the intent classification problem in a Spoken Language Understanding (SLU) setup. We benchmark three types of systems to perform the SLU intent detection task: 1) text-based, 2) lattice-based, and a novel 3) multimodal approach. Our work provides a comprehensive analysis of what could be the achievable performance of different state-of-the-art SLU systems under different circumstances, e.g., automatically- vs. manually-generated transcripts. We evaluate the systems on the publicly available SLURP spoken language resource corpus. Our results indicate that using richer forms of Automatic Speech Recognition (ASR) outputs allows SLU systems to improve in comparison to the 1-best setup (4% relative improvement). However, crossmodal approaches, i.e., learning from acoustic and text embeddings, obtains performance similar to the oracle setup, and a relative improvement of 18% over the 1-best configuration. Thus, crossmodal architectures represent a good alternative to overcome the limitations of working purely automatically generated textual data.
translated by 谷歌翻译
在本文中,我们描述了Case-2022中的子任务2(与休闲新闻语料库的事件因果关系识别)中的共享任务提交。挑战的重点是自动检测新闻媒体中句子中存在的所有因果信号跨度。我们使用T5(一种预先训练的自回归语言模型)检测句子中的因果信号跨度。我们迭代地识别所有原因效应信号跨度三重态,始终在先前预测的三胞胎上预测下一个三重态。为了预测三胞胎本身,我们考虑了不同的因果关系,例如$ \ rightarrow $效果$ \ rightarrow $信号。每个三重态组件都是通过在句子上,当前三重态的前部以及先前预测的三胞胎的语言模型生成的。尽管在一个非常小的160个样本数据集上进行了培训,但我们的方法仍取得了竞争性能,并在比赛中排名第二。此外,我们表明,假设$ \ rightarrow $效果或效果$ \ rightarrow $导致订单实现相似的结果。我们的代码和模型预测将在线发布。
translated by 谷歌翻译